Bayesian recursive variable selection

In this work we introduce a new model space prior for Bayesian variable selection in linear regression. This prior is designed based on a recursive constructive procedure that randomly generates models by including variables in a stagewise fashion. We provide a recipe for carrying out Bayesian variable selection and model averaging using this prior, and show that it possesses several desirable features. In particular, the prior is conjugate—the corresponding posterior is the same stepwise procedure. Moreover, the posterior parameters can be computed analytically through a sequence of recursive computation. This property is particularly desirable in high dimensional settings as it provides an alternative approach for exploring the posterior model space without resorting to Markov Chain Monte Carlo (MCMC), whose convergence behavior is often hard to guarantee and difficult to evaluate in such cases. In addition, the prior also allows flexible ways to incorporate structural features of the model space such as the dependence structure among the predictors. In particular, we illustrate how the prior can be specified to take into account model space redundancy arising from strong correlation among predictors.